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AMENDMENTS TO TRP. DRAWING* ? 

Foimal drawing versions of the figures are submitted herewith. Certain corrections have 
been made to the figures to accord with the substitute specification that has been submitted. 

FIG. 3 has been amended to label the item identified by reference numeral 306 as 
"CORRELATION AND PREDICTIVE STACK RANKING" and to correct a minor 
typographical error. Support for these amendments can be found throughout present application, 
for example at page 13, lines 7-22, of the specification as filed. Accordingly, no new matter has 
been added. 

FIG. 5 has been amended to add reference numerals mentioned in the specification as 
filed, to correct a minor typographical error, and to reroute an arrow from 530 to 534 as 
described in the specification. The figure has also been amended to note that line 548 is 
connected to database 534 as was reflected in the originally submitted specification. No new 
matter has been added. 

FIGS. 6-9 have been amended to remove shading. FIGS. 7 and 9 have also been 
amended to add reference numerals mentioned in the specification as filed. No new matter has 
been added. 

FIGS. 10 and 1 1 have been amended to remove floating boxes objected to in the Office 
Action. FIG. 1 1 has also been amended to correct a minor typographical error. No new matter 
has been added. 

FIG. 12 has been amended to correct a minor typographical error. No new matter has 
been added. 

FIG. 13 has been amended to add reference numerals mentioned in the specification as 
filed. No new matter has been added. 
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REMARKS 

Claims 15-17 (renumbered) are pending. By this Amendment, cIaimslS-17 are amended 
to address certain informalities and typographical errors as noted in tie Office Action. No new 
matter has been added and no narrowing amendments are intended. 

Information Disclosure Statement 
An additional formation Disclosure Statement is submitted herewith citing all of me 
references that have been presented in connection with the prosecution of U.S. Patent 
Application Serial No. 10/410,184. 

Section 102- Ottty 

Claims 15-17 stand rejected as being anticipated by Ostby. This rejection is respectfully 
traversed. 

Ostby is directed to a computerized system operating on a local area network for 
assessing the task-processing style of an individual. Contrary to the suggestion in the Office 
Action, Ostby does not describe connecting the touch sensitive screen as part of a wide area 
network computer system such as the Internet. 

"It should be apparent that computer 12 can comprise a network server and the 
touch-sensitive display screen or other data entry and pointing device can be one 
of a plurality of terminals connected thereto in a local area network." 
(Ostby, Col. 4, lines 4-8, emphasis added). 

There is nothing in Ostby that anticipates the claimed limitation of a plurality of terminals 
connected to the Internet, the limitation of an application screening server connected through the 
Internet to the terminals, or the limitation of a web site identified by a uniform resource locator. 
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More importantly, there is nothing in Ostby that teaches or suggest presenting a set of 
validated questions and, specifically, a short subset of those validated questions to serve as a fast 
job-related prescreen. Contrary to the suggestion in fce Office Action, there is nothing in lines 
12 to 20 of the abstract which supports either the limitation of a set of validated questions, or the 
limitation of a short subset of that set of validated.questions. 

"Each action by the subject undergoing the assessment i s recorded in a raw data 
stream, along with the time that it occurred, and is statistically analyzed with 
respect to several parameters that define the subject's task-processing style. These 
parameters are useful in determining whether an individual is suitable for a job 
and for other assessment purposes, or can be used for training a subject to 
improve the subject's ability and efficiency in dealing with tasks." 
(Ostby, Abstract, 12-20) 

In fact, Ostby teaches away from the concept of using either validated questions or a 
short subset of validated questions because Ostby is set up to record each action of the subject as 
part of an open ended assessment of how the subject response to a hypothetical job situation. In 
this type of open ended assessment, there are no validated questions and answers. 

The subject is informed: "Your task is to take over and do whatever you feel 
needs to be done to meet the demands of the job." It should thus be apparent that 
the task defined by these instructions to the subject Is very loosely structured, 
thereby giving the subject considerable latitude in carrying out the responsibilities 
of the position in completing the simulated task. 
(Ostby, Col. 8, lines 31-39, emphasis added) 

Moreover, only by recording each and every action of the subject can Ostby perform the 
kind of statistical analysis for the specific responses of the given individual who is the subject of 
that test. There is no teaching or suggestion by Ostby of any kind of statistical of a set of 
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question the answers for which are validated against a previously tested control group as a part 
of the analysis of the raw subject data of the given individual in Ostby. 

The step of statistically analyzing the raw subject data includes determining the 
sequential order and frequency of the input and output to and from the individual. 
Eased on this raw subject data, the step of generating the report comprises the 
step of defining a relative order and frequency of input and output by the 

individual 

The output of the individual is characterized as comprising a plurality of types of 
production. The step of statistically analyzing the raw subject data includes 
determining the relative proportion of output from the individual that is of each 
type of production. The types of production preferably include conceptual, 
selective, and implementive. In generating the report, the types of production 
exhibited by the individual in resolving the simulated situation and their relative 
proportion are defined. 

A plurality of options organized as an array on the computer-controlled display 
are presented to the individual. The sequential order in which each of the options 
is selectively accessed by the individual is recorded as part of the raw subject 
data. By analyzing this raw subject data, a determination of the individual's 
methodicalness is determined. The step of generating the report includes the step 
of defining the methodicalness of the individual as one of the preselected 
parameters. 

(Ostby, Col. 4, lines 52-68, Col. 5, lines 1-10, emphasis added) 

The reference made in the Office Action to Column 1, lines 58-67 through Column 2, 
lines 1-4 is inapposite because this is where Ostby is describing other prior art over which Ostby 
distinguishes his invention. Accordingly, it is respectfully submitted that a person skilled in the 
art would not combine the prior art references cited and then dismissed by Ostby as not being 
adequate with the specific teaching of Ostby about the need to analyze the raw data of the given 
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subject in order to determine the methodicalness of that subject in terms of their task-processing 
style. It is respectfully submitted that the argument set forth in the Office Action of a purported 
combination of some general language from the abstract of Ostby with references to other prior 
art discussed in the background section of the O st by patent clearly does not meet the 
requirements of * prima facie case of anticipation, let alone the requirements of an obviousness 
type rejection which was not even made in the Office Action. 

Because Ostby is specifically directed to determining the task-processing style of the one 
subject, there is no need for any kind of rank-ordering. The output of Ostby i s a characterization 
of a single individual in relation to certain predefined criteria. ("In generating the report, the 
types of production exhibited by the individual in ^solving the simulated situation and Iheir 
relative proportion are defined. Ostby, Col. 4, lines 65-68, emphasis added). There is simply 
nothing in the cited portions of Ostby (Col. 4, lines 26-68) which would suggest any need for a 
rank ordering based on an estimation of which of the applicants (plural) have a high probability 
of performing successfully in the job and/or not being terminated early. 

Section 102 - Urn 

Claims 15-17 also stand rejected as being anticipated by Bro. This rejection is also 
respectfully traversed. 

Bro is directed to a computerized system for behavioral and motivational reinforcement 
and guidance. It is respectfully submitted that only by the recitation of the claim language of the 
present invention in the Office Action is there any relevance at all between Bro and the claimed 
invention. While Bro does describe using his behavioral and motivational reinforcement 
guidance techniques with already hired employees (Col, 19, lines 35-43), contrary to the 
suggestion in the Office Action there is nowhere in Bro where his techniques are described in 
any way as a "prediction system for assessing the suitability of job applicants." 
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Unlike Ostby, Bro does include very broad and general language about numerous 
configurations of devices that can include wide area network embodiments for behavioral 
compliance; however, there is absolutely no mention of any kind of database for storing a testing 
program and test data. The portion of Bro cited in the Office Action to support this limitation 
(Col. 8, lines 22-27 and 34 and 35) is, again, referring to some other background prior art and not 
to the invention taught by Bro. More importantly, however, there is no mention anywhere in the 
cited passages of testing program and test da ta 

While Bro may send to his clients certain "polling questions" (Col. 15, lines 14-30) as 
part of determining whether the clients are complying with the behavioral modification system 
that is the focus of the Bro patent, there is simply nothing in Bro which would even remotely 
suggest that these "polling questions" could somehow be interpreted to comprise either the 
recited "requirements questions" or the 'Validated questions" as required by the claims of the 
present invention. The feet that the word "questions" is common to both Bro and the claims of 
the present invention cannot fairly be said to be in anyway provide some kind of teaching or 
suggestion of the very specific claim limitations associated with both the "requirements 
questions" and the 'Validated questions" of the present invention. 

The passing reference to "automatic score keeping" in Bro (e.g. Col 60, lines 3-6) is 
equally unapplicable to the claimed invention. The cited passage from Bro discussing automatic 
score keeping is part of a discussion of the potential applicability of video game-like reward 
systems to the behavioral modification system taught by Bro. Such score keeping reward 
systems are intended to reward an individual player. Absent the impermissible use of the 
hindsight afforded by the claimed invention, it is respectfully submitted that there is simply no 
way that a person skilled in the art would interpret this passing reference to automatic game 
scoring in a video game context as providing any teaching of the kind of scoring and ranking of 
multiple job applicants in accordance with the claimed invention. Likewise, it is unfathomable 
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how this cited passage has any relation to the ability to predict which of multiple job applicants 
have a high probability of performing successfully and not terminating early which is taught and 
claimed by the present invention. 

Similarly, the attempt in the Office Action to characterize items 12 and 16 in Figure 1 as 
teaching the claimed the scoring database and applicant screening server of the present invention 
is an equally unsupportable and untenable stretch of the client database 12 and computer system 
16 taught by Bro. 

It is respectfully submitted that the application of Bro to the claimed inventions can in no 
way support a prima facie case of anticipation or even obviousness of the claimed inventions. 



In view of the foregoing., it is submitted that this application is in condition for allowance. 
Favorable consideration and prompt allowance of the application are respectfully requested. 

The Examiner is invited to telephone the undersigned if the Examiner believes it would 
be useful to advance prosecution. 




Brad Pedersen 
Registration No. 32,432 

Customer No. 24113 

Patterson, Thuente, Skaar & Christensen, PA. 

4800 IDS Center 

80 South 8th Street 

Minneapolis, Minnesota 55402-2100 

Telephone: (612)349-5774 
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Attorney Docket No. 295 1 .03US02 
(SUBSTITUTE SPECIFICATION -- MARKED VERSION) 

COMPUTER-IMPLEMENTED SYSTEM FOR 
HUMAN RESOURCES MANAGEMENT 

Related Application 

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 
60/21 1,044, filed Jun. 12, 2000. 

Background of the Invention 
[0002] To pre vent discrimination in job hiring; and p romoti on, in 1978 the U.S. Federal 
Government established rigorous standards known as t he "Uniform Guidelines" for the use of 
testing and screening instruments used to distinguish a mong candidates for a given position. f41 
CFR §§ 60-3). These standards are designed to p r event testing and screening instruments frnm 
creating an adverse impact on anv m inority or protected groups during the hiring or promotion of 
employees. 

[0003] Over the last twenty y ears, a number of test booklets have been developed and 

published by various publishers in compliance with tha "Uniform Guidelines." One example is 
the Customer Service & Clerical Potent ial Inventory ("CSCPm developed bv Richardson , 
Bellows, Henry and Co.. Inc. The CSCPI is un l ike most other tests in that responses are not right 
or wrong in the tra ditio nal sense. Instead, the performance of the group picking each individual 
item alternative has bee n statisticall y correlated across larp e sa mples of potential and actual 
employees for a position so as to create a test that c a n produce a total score for that p neitin,, 
This total score mav legally be used as a f actor in the hiring or promotion process. 



PAGE 16/131* RCVD AT 1/27/2006 6:04:52 PM [Eastern Standard Time] 1 SVR:USPTO-EFXRF-6/34 1 DNIS:2738300 ' CSID:6123499266 ' DURATION (mm-ss):26-26 



01/27/2006 17:07 FAX 6125499266 



PATTERSON THUENTE SKAAR 



0017/131 



[0004] With the advances in computer systems and the advent of the Internet many 
different aspects of human resources management have been computeriz ed, includin g online 
recruiting. In large p art due to the rigorous standards imposed bv the ''Uniform Guidelines." 
such online recruiting systems use only certain minimum candidate qualifications fc.g.. colleg e 
level degree, possesses a driver's license, number of years experience for a skiin as a wav of 
identifying potential candidates for a position. It has generally been accepted in the human 
resources professio n that choosing among potential candidates based only on minimum 
candidate qualifications will not run afoul of the rigorous standards imposed by the 'Uniform 
Guidelines". 

[0005] Accordingly, it would be desirable to provide a computer-implemented system for 

human resources management that could overcome these limitations and provide further 
advantages in the testing and/or evaluating employees or potential employees. 

Summary of the Invention 
[0006] The present invention is an electronic prediction system for assessing a suitability 
of job applicants fo r an employer. The electronic prediction system includes a plurality of 
terminals connected to the Internet, an applicant screening server connected through the Internet 
to the terminals that has a testing computer program t hat stores test data. A website identified is 
configured to present application questions to the applicants at the terminals and to receive 
applicant responses entered at the terminals in response to presentation of the application 
questions. The application questions include requirements questions eliciting information on 
whether the applicants meet employment requirements and a set of validated questions validated 
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by correlating job performance ratings of a plurality of hired workers with previous responses 
given bv the workers to the application questions before the workers were hired. The set of 
validated questions is a short subset of a large assessment selected to serve as a fast job-related 
pre-screen. A scoring system automatically scores the applicant responses in real timft. The* 
scoring system compares the applicant responses for requirements questions to employer 
requirements and being validated to predict both performance and turnover potential. A scoring 
database is connected to the applicant screening server. An applicant input system located on the 
employer's premises is configured to administer an in-depth assessment to an applicant at the 
employer's premises after the applicant has come to the employer's premises and logged on.^A 
viewing system permits the employer to view applicant results from the electronic prediction 
system and the applicant's rank order, the applicant results providing information on applicants 
who have a high probability of performing successfully and not terminating early 



Brief Description of the Drawings 



[0007] 



FIG. 1 provides a block diagram of an exemplary system in accordance with the 



present invention. 



[0008] 



FIG. 2 illustrates a process for testing and evaluating job applicants in accordance 



with an embodiment of the present invention. 



[0009] 



FIG, 3 depicts a hiring procedure in accordance with one embodiment of the 



invention, 



[0010] 



FIG. 4 is a block diagram of a process employing feedback. 
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[0011] FIG. 5 diagrams an online system in accordance with one embodiment of the 
invention. 

[0012] FIG. 6 shows an example of a web-based presentation for a screening solution. 
[0013] FIG. 7 shows an example of a stack ranked table. 

[0014] FIG. 8 shows an example of a screening solution question presented to an 
applicant taking a screening solution test over the Internet. 

[0015] FIG. 9 shows an example of a structured interview guide for use in an interview 
solution. 

[0016] FIG. 10 illustrates procedural steps that may be followed in a web-based applicant 
system according to an embodiment of the present invention. 

[001 7] FIG. 1 1 illustrates procedural steps that may be followed in a web-based selection 
solution according to an embodiment of the present invention- 

[0018] FIG. 12 illustrates procedural steps that may be followed by an employer 

according to an embodiment of the present invention. 

[0019] FIG. 13 illustrates a human capital management life-cycle. 

Detailed Description of the Drawing s 
[0020] A system for testing a job applicant provides a computerized stack ranking of 
multiple applicants, predictive of the comparative levels of successful job performance. The 
predictive stack ranking may be used as a dynamic interactive filter with a pool of applicants 
over the course of the evaluation or employment process. The system may utilize a 
communications network to communicate between an applicant terminal and a system server. 
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[0021] The system may be used for example for screening, selecting, retaining, 
assigning, or analyzing the job applicant. The job applicant can for example be a new job 
applicant, an employee seeking to retain a job, an employee seeking a different job in the same 
organization, or an employee being evaluated for retention, re-assignment, or promotion. 
Applicants may or may not know they are being evaluated. 

[0022] Once an applicant becomes an employee, the system may collect data regarding 
the employee for use in a feedback loop informing the online hiring process and improving the 
accuracy of the predictive stack ranking. For example, the data may indicate the employer's 
rating of the employee's actual job performance. Such a rating can be cross-checked against the 
answers that the employee gave during the application process. The cross-checking can be used 
as feedback to refine the questions and evaluation criteria used at each stage of the hiring 
process. For example, the cross-checking may be analyzed to select from among many questions, 
a small subset having high predictive value. The small subset can then be used in a quick initial 
screening stage. Or, the small subset can be given greater weight than other questions in a 
computerized stack ranking of candidates. 

[0023] FIG. 1 provides a block diagram of an exemplary system in accordance with the 
present invention. A job applicant can use applicant terminal 102 to communicate over network 
1 04 with system server 106. Applicant terminal 102 may for example be a telephone handset, a 
personal computer, a workstation, a handheld wireless device such as those marketed under the 
trademarks PALM or HANDSPRING, or a Wireless Application Protocol enabled device such 
as a mobile phone. Network 104 may for example be the Internet, the World Wide Web, a wide 
area network, a local area network, a telephone network, a wireless communication network, a 
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combination thereof, or any other link capable of carrying communications between an applicant 
terminal and a server. 

[0024] System server 106 employs a testing computer program 108 and has access to a 
scoring database 110. System server 106 communicates with applicant terminal 102 in 
accordance with instructions from testing computer program 108. 

[0025] System server 106 may communicate with employer server 112 over network 104 
or over direct link 1 14. System server 106 is shown a$ a unitary server, but may be a distributed 
computing platform. 

[0026] An applicant terminal may be remote from, or co-located with, system server 106 

and/or employer server 112, For example, applicant terminal 102 may be located at a job 
applicant's home, applicant terminal 1 16 may be located at a job fair or employment office, and 
applicant terminal 120 may be located at an employer's location. 

[0027] Partner server 121 may be linked to network 104 and system server 106 to 
facilitate integration of a business partner seeking to participate in the system of FIG. 1 . 
[0028] System server 106 may pose questions to a job applicant located at an applicant 
terminal, receive responses from the job applicant, and score the answers in accordance with 
scoring database 110. The scoring may take place in real time, i.e., while the applicant is still 
online, and may be reported in the form of a comparative stack ranking of multiple applicants. 
The stack ranking may be delivered from system server 106, over either network 104 or direct 
link 1 14, to employer server 1 12. 

[0029] Scoring of each answer by system server 106 may be instant, i.e., before the next 

question is answered. Thus, adaptive testing techniques may be implemented over network 104. 

For example, the answers given by an applicant at applicant terminal 102 to questions 
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propounded early in a test may determine which questions are propounded by system server 106 
to the applicant later in the same test. In addition, if an applicant at terminal 102 provides an 
unacceptable answer to a disqualifying "knock-out" question, server 106 may immediately 
terminate the test. 

[0030] These same adaptive testing principles may be applied to a software program used 
to support a real time interview, either in person or over a communications network. For 
example, an employer conducting an oral interview in person or over a telephone can enter a 
candidate's oral answer into employer terminal 124, which then communicates the answer to 
system server 106, which in turn suggests via employer terminal 124 the next question for the 
employer to ask the interviewee. 

[0031] The system may test an online applicant for any competency desired, in any 
sequence. The tested competencies may be abilities, traits, knowledge, skills, etc., that have been 
proven relevant to and predictive of successful job performance. By way of example and not 
limitation, the following competencies may be tested: 



[0032] 


I. 


dependability 


[0033] 


2. 


agreeableness 


[0034] 


3. 


critical thinking 


[0035] 


4. 


problem solving ability 


[0036] 


5. 


talkativeness 


[0037] 


6. 


assertiveness 


[0038] 


7. 


gregariousness 


[0039] 


8. 


persuasiveness 
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[0040] 


9. 


achievement 


[0041] 


10. 


education 


[0042] 


11. 


experience 


[0043] 


12. 


customer service orientation 


[0044] 


13. 


customer focus 


[0045] 


14. 


conscientiousness 


[0046] 


15. 


self-confidence 


[0047] 


16. 


motivation 


[0048] 


17. 


revenue focus 


[0049] 


IS. 


cognitive ability 


[0050] 


19. 


leadership 


[0051] 


20. 


decision making 


[0052] 


21. 


flexibility 


[0053] 


22. 


commitment 


[0054] 


23. 


learning ability 


[0055] 


24. 


dedication 


[0056] 


25. 


tenacity 


[0057] 


26. 


number of jobs held 


[0058] 


27. 


length of time in job(s) 


[0059] 


28. 


working with information 


[0060] 


29. 


supervisory potential 



8 



PAGE 23/131 * RCVDAT 1127/2006 6:04:52 PM [Eastern Standard Time] ' SVR:USPTO-EFXRF-6!34 s D HIS: 2738300 ' CSID:6123499266 ' DURATION (mm-ss):26-26 



01/27/2006 17:09 FAX 6123499266 



PATTERSON THUENTE SKAAR 



E1024/131 



[0061] 


30. 


judgment 


[0062] 


31. 


leadership 


[0063] 


32. 


coaching skills 


[0064] 


33. 


teamwork 


[0065] 


34. 


interpersonal skills 


[0066] 


35. 


business leadership 


[0067] 


36. 


leadership motivation 


[0068] 


37. 


self-leadership 


[0069] 


38. 


interpersonal leadership 


[0070] 


39. 


communication skills 


[0071] 


40. 


management potential 


[0072] 


41. 


likelihood of retention 


[0073] 


42. 


self-control 


[0074] 


43. 


energy 


[0075] 


44. 


executive potential 


[0076] 


45. 


listening orientation 


[0077] 


46. 


language skills (English, etc.) 


[0078] 


47. 


verbal reasoning 


[0079] 


48. 


spatial ability 


[0080] 


49. 


interest 


[0081] 


50. 


motivation 
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[0082] Typically, system server 106 tests for certain ones of the competencies that have 
been proven to be predictive of successful performance of the type of job for which the applicant 
is being considered. The results of the testing are tabulated in a stack ranked table. The stack 
ranked table may rank a number of applicants against each other and list them in order, from first 
to last. The table may also present other information for each applicant. The other information 
may include, by way of example and not limitation: 
[0083] L Name 

[0084] 2. Identifying number (e.g. social security number)* 

[0085] 3 . Score achieved at various stages for various competencies. 

[0086] 4. Recommendation (or not) to continue the hiring process beyond each stage 

[0087] 5. Link to application information (e.g. address, Tesume details) 

[0088] 6. Contact information (phone number, e-mail address, mailing address, etc.) 

[0089] 7- Date of application 

[0090] 8* Success or failure in complying with knockout requirements for the job 
[0091] 9. Screening solution scores, presented a$ percentiles 

[0092] 10. A calculated recommendation to proceed or not to proceed with the 

applicant 

[0093] 1 1 . Results (by competency) of the selection solution 

[0094] 12. Link to allow manual entry of the test answers if not done on computer 
directly by the applicant 
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[0095] 13. A calculated recommendation to hire or not hire based on a weighted 
overall score of selection competencies (or other factors the hiring company wishes to use and 
that are approved as statistically valid and legally defensible) 

[0096] 14. Additional columns for storage of data from a structured behavioral 
interview 

[0097] 15. Additional columns for storage of data from other decision-making 
processes such as drug testing, reference checks, or medical exams. 

[0098] A process for testing and evaluating job applicants may be described with 
reference to PIG. 2. Generally, applicant testing 201 includes providing a test to a job applicant 
and scoring the applicant's answers. The test may be administered online or it may be 
administered manually off-line. Scores are entered into a system for calculating a stack ranked 
table. Predictive stack ranking 202 generally includes ranking a job applicant against other job 
applicants in order from first to last or other comparative ranking. The other job applicants may 
be current job applicants, past job applicants, or fictional job applicants. 

[0099] FIG. 3 depicts a hiring procedure in accordance with one embodiment of the 

invention. Announcement 302 may be an online job announcement such as a web page with an 

"apply now" hyperlink icon. The web page may reside on an employer's website or an 

employment agency website, for example. Or, an online job announcement may be a recorded 

announcement on a menu-driven telephone voice processing system. Alternatively, 

announcement 302 may be an offline job announcement such as a newspaper advertisement, 

[00100] In response to announcement 302, an interested job applicant requests 

adrninistration of screening test 304. Screening test 304 may be remotely administered and 

scored online, i.e., unproctored, with the scores being automatically provided to correlation and 

11 
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predictive stack ranking 306, Alternatively, screening test 304 may be administered manually 
with paper and pencil, and then graded by hand or machine, with the scores being provided to 
correlation and predictive stack ranking 306. The predictive stack ranking may for example be 
constructed by system server 106 or employer server 112. 

[00101] Correlation and predictive Prodiotivo stack ranking 306 totals the graded answers 
according to particular competencies known to be relevant to successful job performance. In a 
preferred embodiment the questions to be asked at the various stages are selected for a particular 
type of job being offered in accordance with a proven relationship with desired business 
outcomes. Business outcomes can for example include: level of sales, customer satisfaction. 
quality m easures such as fault rates, retention and tenure of employment time keening, learning 
ability, progression to more senior roles over time, and supervisor ratings of behavioral success 1 
The particular type of fob is defined in conjunction with the U.S. Department of Labor "Q*NET" 
classification system. Some types of jobs might include customer service, technical, professional, 
or managerial Various competencies are determined to be associated with desired business 
outcomes for a given type of job. These competencies are tested for at various solution stages 
with appropriate Questions. 

[0104] The appropriate competencies, questions, scoring, weighting, and ranking factors 

for a new job can be designed from historical tests for existing jobs, by applying statistical 
techniques and using the gathering of data on the Internet to ensure rapid validation of the new 
assessment solution. Confirmatory job analysis is used to determine the appropriateness of 
solutions for a particular job. 

[00102] Correlation and predictive Prcdiotivc stack ranking 306 may be administered by a 

computer processor located at system server 106, for example. Predictive stack Tanking 306 may 
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give different weight to different questions, and may at any stage immediately disqualify an 
applicant providing an unacceptable answeT to a "knock-out' 1 question. Correlation and 
predictive Pr e dictiv e stack ranking 306 may rank the applicant in order against other job 
applicants in a table. Correlation and predictive Predictiv e stack ranking 306 may be used to 
decide which applicants to invite for the next stage, selection test 308, 

[00103] Selection test 308 is preferably conducted under supervised conditions , i.e., 
proctored . For example, selection test 308 may be administered in person. An in-person test may 
take place at a job fair, an employer's location, a job site, or an employment agency. An in- 
person test may include verification of the job applicant's identity, such as by examination of a 
photo identification document produced by a test-taker. Selection test 308 may be administered 
dnline or manually. Supervised conditions typically include observation of the test-taker during 
administration of the test The answers to selection test 308 are graded and the results are 
incorporated in correlation and predictive stack ranking 306. 

[00104] Correlation and predictive Pr e dictiv e stack ranking 306 may then update a 
previously created entry for the applicant and rank or re-rank the applicant in order against other 
job applicants. After this is accomplished, the highest ranking applicants may be invited for 
interview 310. 

[00105] Interview 310 may be structured or unstructured, online or in person. If interview 

310 is structured, a program leads the interviewer through the interview by suggesting questions 

one at a time. The program may be a list of questions written on paper or it may be a computer 

program resident for example in system server 106. The program suggests questions that are 

predetermined to be valid, i.e., proven to be associated with successful job performance and 

legally permitted. The interviewer can input the answers and/or a score for the answers, either 

13 
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after each answer or at the conclusion of the interview. This can be done via employer terminal 
124, for example. 

[00106] Interview 310 results in an interview score being provided to correlation and 
predictive stack ranking 306. Correlation and predictive Predictive stack ranking 306 is revised 
to reflect the interview score. In particular, the relative rank of the job applicants is reassessed, 
[00107] FIG. 4 is a block diagram of a process employing feedback. Test design 402 is 
initially performed using industry-accepted standards. Test administration 404 tests and scores 
job applicants and/or incumbents. Employee performance evaluation 406 measures actual job 
performance of the applicant or incumbent after holding the job for a period of time. This 
information is fed back to test design 402 and/or test administration 404. Test design 402 may be 
revised to delete questions which were not predictive of successful job performance. This can be 
done for example by deleting questions whose answers bore no relation to performance 
evaluation 406 for a statistically valid sample. Test administration 404 may be revised by 
adjusting the weight given to certain questions or answers that showed an especially strong 
correlation to employee performance evaluation 406, For example, if test administration 404 is 
associated with correlation and predictive stack ranking 306, feedback from employee 
performance evaluation 406 may help determine how various job applicants are comparatively 
ranked against each other. 

[00108] FIG. 5 diagrams an online computer based system 500 in accordance with one 
embodiment of the invention. Box 502 represents a job vacancy with a requirement for an online 
screening and selection solution. The vacancy can come to the attention of a potential job 
applicant in a number of ways. 

14 
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[001 09] For example, box 504 represents an online application via a hiring company's own 
website. A company offering a job may post a vacancy announcement on the company's website 
and invite job seekers to apply by clicking on an icon labeled "apply here" or the like. Box 506 
represents a similar posting on an online job board. Box 508 represents candidates given a 
Uniform Resource Locator (URL) directly by the company. This may occur when the company 
offering a job identifies a potential candidate. Box 510 represents a media advertisement 
including a URL for a job. Thus, job seekers observing the advertisement can direct their 
browsers to the indicated URL. 

[00110) At job fair 512, job seekers may be provided a URL associated with the company 
or the particular vacancy. Paper-and-pencil measures could also be used at job fairs and entered 
into the system. A computer terminal may be provided for use of job seekers at job fair 512, 
enabling job seekers to participate in the online system. Box 514 represents an executive search 
via a recruiter network. Job seekers relevant to the search are identified in recruitment firm 
applicant database 516. Database 516 can link to a URL associated with the job. 
[001 1 1] Preferably, no matter how a potential applicant becomes aware of or identified for 
a job opening in system 500, the potential applicant is considered at decision 520. Decision 520 
asks whether applicant has completed the required screening solution 524. If not, the applicant at 
box 522 is given via e-mail, mail, or in person, a URL for assessment. For example, system 500 
may send an e-mail message to a potential applicant, the e-mail message inviting the potential 
applicant to apply for vacancy 502 by directing a browser to a screening solution URL provided 
in the e-mail message. Alternatively, when a potential applicant is visiting a website at which 
decision 520 determines that the required screening solution has not been completed, the website 
host can provide a link to a web page identified by the screening solution URL. Decision 520 
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may be based on a potential applicant's name, e-mail address, and/or other identifying 
information. 

[001121 Screening solution 524 is administered via the Internet and is hosted at the 
screening solution URL mentioned above. Screening solution 524 asks screening questions to 
ascertain if the applicant has the basic qualifications to do the job. These are based on questions 
typically asked by recruiters but which are statistically validated over time to ensure they are 
legally defensible and predictive. The questions may include a combination of biodata and 
personality measures. They may include self-assessments of skill levels appropriate to the job 
requirements. Screening solution 524 requires applicants to transmit elicited information over the 
Internet A possible example of a web-based presentation for screening solution 524 is illustrated 
in FIG. 6. Screen shot 600 shows a portion of the presentation. 

[00113] Once completed, screening solution 524 provides applicant feedback 540 and 
conveys applicant details and screening scores to stack ranked table of applicants 530, Applicant 
feedback 540 may provide a message to the online applicant indicating that the screening 
solution is complete, that the applicant has passed or failed the screening stage, and that the 
applicant may or may not be contacted in due course. Other information may also be provided to 
the applicant in the feedback pages, like a realistic job preview, recruiter phone number, 
scheduling information, etc. 

[00114] Once an applicant has completed the screening solution, system 500 ranks the 
applicant in comparative order against other applicants in stack ranked table of applicants 530. A 
certain number or percentage of applicants in table 530 will be chosen for further consideration. 
For example, the applicants ranking among the top five of all applicants ranked in table 530 may 
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be chosen for advancement in the system at this juncture. Information identifying the chosen 
applicants will be included on a "short list" as indicated by box 536, 

[00115] The short list chosen at box 536 is transmitted to selection solution 538, at which 
the advancing applicants are invited to answer selection questions. Selection solution 538 asks 
additional questions and requires an advancing applicant to input answers. Preferably, the 
applicant completes selection solution 538 while sitting at a tenninal located at one of the 
company's locations. The terminal communicates over the Internet with a website set up to 
administer the selection solution. 

[00116] At the conclusion of selection solution 53 8, applicant feedback 540 is provided 
from the website to the applicant, and applicant details and scores 541 are incorporated in stack 
ranked table 530, Feedback 540 may optionally include a sophisticated report on the applicant's 
strengths and weakness. The applicant may then be directed to an appropriate web page chosen 
by the hiring company. One page may indicated successful completion and a second page may 
indicate failure. The appropriate web page may suggest other openings appropriate to the 
applicant's test responses and may provide hyperlinks the applicant can use to initiate the 
application process for these other openings. 

[00117] Once stack ranked table 530 re-ranks the applicants as a result of selection 
solution 538, some applicants are invited to participate in interview solution 542. For example, 
the top three applicants as ranked by table 530 after selection solution 538 may be invited for an 
in-person interview. Because the selection solution is preferably in instant communication with 
stack ranked table 530, the interview invitation may be extended immediately at the conclusion 
of the selection solution. 
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[00118) Interview solution 542 is preferably a structured interview, with questions 
provided via the Internet to the interviewer at the company's location. The interviewer reads the 
provided questions and reports a score over the Internet from the company's location for 
incorporation in stack ranked table 530. Benchmark performance anchors may assist the 
interviewer in grading the applicant's responses. 

[00119) Interview solution 542 can be designed according two exemplary models. In a 
first model, an employer is provided with standard interview guides for several job types as well 
as the competency templates for these types so that the employer can build variations to meet 
specific needs. In a second model, an employer can build new interview guides and new 
competency templates. In the second model, the employer has access to the full array of work- 
related competencies and associated questions in a comprehensive question bank. 

[00120] In ranking applicants, stack ranked table 530 may consider a combination of 
different biographical, personality, behavioral, and other appropriate information and 
competencies. In addition to the comparative ranking, table 530 may indicate for each applicant a 
yes/no recommendation, a percentage likelihood of successful job performance, biographical 
information not used for evaluative purposes, and so forth. 

[00121] Stack ranked table 530 may be developed by grading the various solution stages 

with a computer implementing the following algorithm. First, search for disqualifying answers to 

"knock-out" questions. Second, give points for answers matching those of the previously hired 

candidates who achieved a successful performance evaluation. Third, deduct points for answers 

matching those of the previously hired candidates who received an unsuccessful performance 

rating. Fourth, multiply the added or subtracted points by any weighting assigned each question. 

Fifth, sum the points for all questions related to a given competency. Sixth, compare the summed 
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points for each competency to norms of either the job-holders in the company or a wider 
population. Seventh, predict performance of the applicant as a worker in the job, based on the 
business outcomes identified by the hiring company and the competencies that contribute to 
those outcomes. 

[00122] A final selection 546 is made based on stack ranked table 530. Preferably, the 
selection is transmitted over the Internet to the company, enabling the company to make an offer 
to the selected applicants). For example, if there is only one opening, an offer may be extended 
to the applicant ranked highest by stack ranked table 530. If the applicant accepts the offer, the 
applicant is employed by the company. If the applicant declines, the next highest ranked 
applicant in stack ranked table 530 is offered the job. If a plural number of openings exist, that 
number of applicants may be selected off the top of stack ranked table 530 and offered the job. If 
one of the applicants declines, the next highest ranked applicant in stack ranked table 530 is 
offered the job. Data from stack ranked table 530 is forwarded to data warehouse 534. 
[00123] The performance of successful applicants is monitored during their employment. 
At box 550, performance data for successful applicants are collected at a later date, and sent 548 
to data warehouse 534. 

[00124] Data collected at data warehouse 534 are used for research and development and 
for reporting purposes. For example, functions enabled by storing comprehensive data generated 
by system 500 may include: 

[00125] Storage of question level responses from applicants for jobs. This can be used for 
re-checking of applicant information (auditing etc.) and for research to develop new solutions 
and questions. 
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[00126] Reporting on Equal Employment Opportunity Commission requirements. Data on 
ethnicity etc. can be stored to enable an employer to comply with reporting requirements to 
government agencies, 

[001271 Source of data for designing new solutions including data on the nature of the job 
and the competencies that are required by the role Gob analysis). This data is collected using 
online assessments. 

[00128] Source of data for statistical research on correlation between the solutions and 
their predicted outcomes for applicants, and the actual outcomes for employees who were hired 
(validation studies). 

[00129] Design of solutions other than recruitment related solutions. 

[00130] Reporting of usage volumes for billing and financing accounting purposes. 

[00131] Because system 500 preferably uses instant communications, adaptive testing 

techniques may be implemented online. An applicant's failure to overcome hurdles in a given 

solution will deliver a different path through the solution than that of a successful applicant- The 

degree of advancement of a given applicant through system 500 may result in different charges 

to the company from a solutions provider. For example, a solutions provider that hosts a website 

supporting screening solution 524, selection solution 538, and interview solution 542 may charge 

the hiring company the following amounts: one dollar for every applicant completing only the 

screening solution, five dollars for every applicant advancing only to the end of the selection 

solution, ten dollars for every applicant rejected after the interview solution, twenty dollars for 

every applicant offered a job, and fifty dollars for every applicant accepting an offer. 

[00132] In practice 3 any of the various stages (screening solution 524, selection solution 

538, and interview solution 542) may be skipped, re-ordered, combined with other stages, or 
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eliminated. Or, a short telephone interview may be structured early in the process to quickly 
screen applicants. 

[00133 } In a preferred embodim e nt, the questions to b e ask e d at the various stages ar e 

selected for a particular type of job being off e red in accordance) with a prov e n r e lationship with 
d e sir e d business outcomes. Bia s in e ss outcome s can for exampl e includ e : lev e l of sales, customer 
satisfaction, quality m e a s ure s ouch aa fault rate s , retention and t e nur e of e mployment, time 
keeping, l e arning ability, progression to mor e s e nior roles ovor - tim e , and sup e rvisor ratings of 
b e havioral success. - Th e particular typ e of job is d e fin e d in conjunction with the U.S. Department 
of Labor n O*NET" olasoifioation syst e m. Som e types of jobs might includ e customer s e rvice, 
technical, professional, or manag e rial. Various competenci es ar e det e rmined to bo associat e d 
with d e sired bu s iness outcom e s for a given typo of job. Thes e compet e ncies - aro tested for at 
various solution stages with appropriat e questions ? 

[0013 4 ] Th e appropriat e competencies, qu es tions, scoring, weighting, and ranking factor s 

for a n e w job eon bo design e d from historical-toots for existing jobs, by applying statistical 
t e chniques ond using the gathering of data on - tho Intomot to ensur e rapid validation of th e new 
ass e ssment solution. Confirmatory job analysis - is UGod to d e t e rmin e th e appropriateness of 
solutions for a p articular job. 

[00135] FIG. 7 shows an example of a stack ranked table. Computer screen shot 700 
illustrates a sample stack ranked table 730 for a customer service job. Various tabs permit 
viewing of data generated by each solution stage. Tab 702 reveals data 703 from a screening 
solution, tab 704 reveals data 705 from a selection solution, tab 706 reveals data 707 from an 
interview solution, and tab 708 reveals all results. In screen shot 700, tab 708 is selected. 
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[00136] Section 709 of screen shot 700 shows general information about each applicant, 
including current rank 710, a link 712 to application information (not shown), last name 714, 
first name 716, and application date 718. 

[00137J Screening solution data 703 includes an indication 720 of whether each applicant 
successfully passed the knockout requirements for the job. Data 703 also includes scores on 
certain competencies such as educational and work related experience 722, customer service 
orientation 724, and self-confidence 726. Column 728 indicates whether each applicant is 
recommended to advance beyond the screening stage. 

Selection solution data 705 includes scores on certain competencies such as customer focus 732, 
conscientiousness 734, and problem solving 736. Column 738 indicates whether each applicant 
is recommended to advance beyond the selection stage, finliimn 7 40 includes a detailed report 
for each applicant wh ^ nfthwrm 750 includes a score. 

[00138] Additional information (not shown) may include columns for storage of data from 
other decision-making processes such as drug testing, reference checks, or medical exams. 
[00139] FIG. 8 shows an example of a screening solution question presented to an 
applicant taking a screening solution test over the Internet. In screen shot 800, simulated 
customer contact record 802 is presented to the applicant. The applicant is asked question 804, 
and is required to click on a circle next to one of the answers. Question 804 may test for a 
competency in working with information, for example. 

[00140] FIG. 9 shows an example of a structured interview guide for use in an interview 

solution. As illustrated, the interview guide is being presented online on a computer screen to an 

interviewer conducting an interview with an applicant. Screen shot 900 shows interview item 

902 for a sample customer service job. The customer service job opening is for a call center 
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position, and revenue focus has been identified as a relevant and predictive competency. Item 
902 elicits from the applicant a situation 904, the applicant's behavior 906 in the situation, and 
the outcome 908 reported by the applicant. The interviewer can grade the applicant's responses to 
item 902 by marking a score 910 from 1 to 10, 

[00141] FIG. 1 0 illustrates procedural steps that may be followed in a web-based applicant 
system according to an embodiment of the present invention. 

[00142] FIG. 1 1 illustrates procedural steps that may be followed in a web-based selection 
solution according to an embodiment of the present invention. For example, these steps may 
follow those illustrated in FIG, 1 0- 

[00143] FIG. 12 illustrates procedural steps that may be followed by an employer 
according to an embodiment of the present invention. 

[00144] The following tables provide examples of screening solutions and selection 
solutions designed for different types of jobs. The tables show components (competencies) 
shown to be relevant to successful performance of each job type. In the tables, some components 
axe considered required, and others are considered optional. 



[00145] 



Table One: Entry/General Skilled Solutions 




Solution 


Definition 


Items 




Component 






Screening 7-10 Minutes 
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Required 


Educational and 
Work-Related 
Experience 


Measures potential for 
success in entry-level jobs 
across industry type and 
functional area. Scores on 
Education and Work-Related 
Experience are derived from 
candidates' responses to 
questions regarding 
developmental influences, 

work-related values and 
attitudes- 


15 


Self-Confidence 


This component references: 
belief in one's own abilities 
and skills and a tendency to 
feel competent in several 
areas. 


7 


Optional 


Decision 
Making/Flexibility 


Measures potential for 
success in entry level 
positions. Scores on 
Decision Making and 
Flexibility are derived from 
candidates 5 responses to 
questions regarding 
developmental influences, 
self-esteem, work history and 
work-related values and 
attitudes. 


8 


Screening 23 -35 Minutes 


Required 


Conscientiousness 


This component is designed 
to predict the likelihood that 
candidates will follow 
company policies exactly, 
work in an organized 
manner, return from meals 
and breaks in the allotted 
time, and keep working, even 
when coworkers are not 
working. 


65 


Retention Predictor 


Measures commitment, 
impulsiveness, responsibility, 
and motivation. It predicts 
the likelihood that new hire 
will remain on the job for at 
least three months. 


44 
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Optional 


Leaning Ability 

j 


1 his component measures 
the tendency to efficiently 
and effectively use numerical 
and analytical reasoning. 
This competency is 
characterized by the ability to 
leam work-related tasks, 
I processes, and Dolicies. 


54 

(12 minute timer) 


[00146] 




Screening 8 - ] 


Solution 
Component 
0 Minutes 


^uoLunici ocrvicc Lsoiuuon 

Definition 


Items 


Required 


Educational and 
Work-Related 
Experience 


Measures potential for 
success in customer service 
jobs. Scores on Education 
and Work-Related 
Experience are derived from 
candidates responses to 
questions regarding 
developmental influences, 
self-esteem, work history and 
work-related values and 
attitudes. 


15 


Customer Service 
Orientation 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
sincerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


20 



25 
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Optional 


Self-Confidence 


'His component references; 
belief in one's own abilities 
and skills and a tendency to 
feel competent in several 
areas. 


7 
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RftrRRnina 1 7 - 29 - 27 Minutes 




— — ■ 


"R pfimrfid 


Customer Focus 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
incerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


32 




Conscientiousness 


This component is designed 
to predict the IiKeiinooa that 
candidates will follow 
company policies exactly, 
work in an organized 
manner, return from meals 
and breaks in the allotted 
time, and keep working, even 
when coworkers are not 
working. 


65 


Optional 


Learning Ability 


This component measures 
the tendency to efficiently 
and effectively use numerical 
and analytical reasoning. 
This competency is 
characterized by the ability to 
learn work-related tasks, 
processes, and policies. 


54 

(12 minute timer) 


Optional 


Retention Predictor 


Measures commitment, 
impulsiveness, responsibility, 
and motivation. It predicts 
the likelihood that a new hire 
will remain on the job for at 
least three months 


44 

1 
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Table Three - Customer Service Solution: Sales Positions 





Solution 
Component 


Definition 


Items 


Screening 9-15 Minutes 


Required 


Educational and 
Work-Related 
Experience 


Measures potential for 
success in customer service 
jobs. Scores on Education 
and Work-Related 
Experience are derived from 
candidates responses to 
questions regarding 
developmental influences, 

coif. ^ctppm wrtTV Vitctrvrv j^nn 

work-related values and 
attitudes. 


15 


Customer Service 
Orientation 


This component is designed 
to predict the likelihood that 
candidates will show 
persistent enthusiasm in 
customer interaction, 
apologize sincerely for 
inconveniences to customers, 
be patient with customers, 
tolerate rude customers 
calmly, and search for 
information or products for 
customers. 


20 


Optional 


Sales Potential 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


23 

■ 
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Screening 1 5- 27 Minutes 


Required 


Sales Potential 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


60 




Customer Focus 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
sincerely inconveniences to 
customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


32 


Optional 


Learning Ability 


This component measures 
the tendency to efficiently 
and effectively use numerical 
and analytical reasoning. 
This competency is 
characterized by the ability to 
learn work-related tasks, 
processes, and policies. 


54 

(12 minute timer) 
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Table Four - Customer Service Solution: Call Center Positions 




Solution 
Component 


Definition 


Items 


Screening 9 -1 


1 minutes 


Required 


Educational and 
Work-Related 
Experience 


1 Measures potential for 
success in customer service 
jobs. Scores on Education 
and Work-Related 
Experience are derived from 
candidates responses to 
questions regarding 
developmental influences, 
self-esteem, "work history and 
work-related values and 
attitudes. 


15 


Customer Service 
Orientation 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction- anoloffize 
sincerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


20 


Optional 


Self-Confidence 


This component references: 
belief in one's own abilities 
and skills and a tendency to 
feel competent in several 
areas. 


7 
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Screening 16- 


-31 -39Mintues 






Required 


Customer Focus 


This component is designed 
to predict the likelihood that 
candidates will show 
persistent enthusiasm in 
customer interaction, 
apologize sincerely for 
inconveniences to customer^ 
be patient with customers, 
tolerate rude customers 
calmly, and search for 
information or products for 
customers. 


32 




Conscientiousness 


This component is designed 
to predict the likelihood that 
candidates will follow 
company policies exactly, 
work in an organized 
manner, return from meals 
and breaks in the allotted 
time, and keep working, even 
when coworkers are not 
working. 


65 




Working with 
information 


This component is designed 
to predict success in 
customer service call-center 
jobs by assessing a 
candidate's ability to retrieve 
information and use it in 
order to solve problems. 


30 

(15 minutes timer) 


Optional 


Retention Predictor 


Measures commitment, 
impulsiveness, responsibility, 
and motivation. It predicts 
the likelihood that a new hire 
will remain on the job for at 
least three months. 


44 
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Tabl 


e Five -Customer Service Solution: Call Tenter SI^Ipq Potions 




Solution 
Component 


Definition 


Items 


Screening 9 - 


15 Minutes 


Required 


Educational and 
Work-Related 
Experience 


Measures potential for 
success in customer service 
jobs. Scores on Education 
and Work-Related 
Experience are derived from 
candidates' responses to 
questions regarding 
developmental influences, 
self-esteem, work history and 
work-related values and 
attitudes. 


15 


Customer Service 
Orientation 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
sincerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers cainiiy* and search 
fOT information or products 
for customers. 


20 


Optional 


Sales Potential 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


23 
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Screening 30 1 
Required 


Minutes 

Sales Focus 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale,, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


60 


Customer Focus 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
sincerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


32 




Working with 
Information 


This component is designed 
to predict success in 
customer service call-center 
jobs by assessing a 
candidate's ability to retrieve 
information and use it in 
order to solve problems. 


30 

(15 minute timer) 
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Table Six -Sales Solutions 




Solution 
Component 


Definition 


Items 


Screening 10- 


- 14 Minutes 




Required 


Educational and 
Work-Related 
Experience 


Measures potential for 
success in customer service 
jobs. Scores on Education 
and Work-Related 
Experience are derived from 
candidates responses to 
questions regarding 
development influences, self- 
esteem, work history and 
work-related values and 
attitudes. 


15 




Sales Potential 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


23 


Optional 


Customer Service 
Orientation 


Designed to predict the 
likelihood that candidates 
will show persistent 
enthusiasm in customer 
interaction, apologize 
sincerely for inconveniences 
to customers, be patient with 
customers, tolerate rude 
customers calmly, and search 
for information or products 
for customers. 


20 
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Screening 1 0 - 25 - 40 Minutes 


Required 


Sales Focus 


Designed to predict the 
likelihood that candidates 
will suggest or show 
alternative solutions based on 
customer needs, direct 
conversation toward a 
commitment/ order/ sale, 
show confidence even after a 
hard refusal/rejection, and 
strive to close a transaction 
every time. 


60 


Optional 


Problem Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. This competency 
is characterized by the ability 
lu duiYc tHJiiipicA prooierns, 
and make reasoned decisions. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning. This 
competency is characterized 
by the ability to verbally 
explain complex information 
to others. 


10 
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Table Sevei 
1 Solution 
1 Component 
Screening 1 0 - 20 Minute 


i- Supervisory Solutions 
Definition 


Items 


Required 


Supervisory 
Potential 


Measures tjotentia] fnr 
supervisory success across 
industry type and functional 
area. Scores on Supervisory 
Potential are derived from 
candidates' responses to 
questions regarding 
academic and social 
background, and aspirations 
concerning work. 


10 


Judgment 


Alea&lireS tintf^tvHsil fnr 
invu^wiu IJVj LtUuloJ, XOr 

making good judgments 
about how to effectively 
respond to work situations. 
Scores on Judgment are 
derived from candidates' 
responses to questions 
regarding situations one 
would likely encounter as a 
manager/supervisor. 


10 


Optional 


Leadership/Coaching 

Teamwork/ 
Interpersonal Skills 


Measures potential for 
success as a supervisor. This 
is done by having applicants' 
make judgments about the 
most effective teamwork and 
leadership behaviors in 
specific work situations. 
Scores are determined by 
comparing their response 
profiles to the profiles of 
supervisors who are known 
to be successful. 


19 
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Screening 22 - 
Required 


-37*52 Mins 
Business 
Leadership 


Measures the candidate's 
thinking styles. High scorers 
are likely to have or learn 
good planning and 
organizing skills, be 
innovative, consider issues 
from multiple perspectives, 
and create strategies to build 
their business. 


28 


Required 


Leadership 
Motivation 


Measures the candidate's 
desire for achievement, 
drive, initiative, energy level, 
willingness to take charge, 
and persistence. High 
scorers are likely to be highly 
motivated to succeed and to 
set challenging goals for 
themselves and othere. 


23 




Self-Leadership 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility for actions, and 
tolerate stress. High scorers 
are also likely to have a 
positive attitude, be 
optimistic about the future, 
and demonstrate high levels 
of professionalism. 


32 


Screening 22 - 37 - 52 Mins Ccont) 




Interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 
persuade and influence 
others, gain commitment, 
and build effective 
interpersonal relationships. 
They also have potential to 
develop skills in the area of 
employee relations, 
coaching, motivating, and 
leading a team. 


30 
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Optional 


Decision 
Making/Problem 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. This competency 
is characterized by the ability 
to solve complex problems, 
and make reasoned decisions. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning. This 
competency is characteri2ed 
by the ability to verbally 
explain complex information 
to others. 


10 


[00152] 


Table Eight - Professional Solutions 




Solution 
Component 


Definition 


Items 


Screening 7 - Minutes 


Required 


Dependability 


This competency is 
characterized to behave in 
expected and agree upon 
ways; following through on 
assignments and 
commitments; keep 
promises; and accept the 
consequences of one's own 
actions. 


40 




Interpersonal Skills 


This competency is indexed 
by a tendency to be pleasant, 
cooperative, and helpful 
when working with others, as 
well as flexible in conflict 
resolution situations. 





38 
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Selection 35 - 


Self-Control 


This competency is 
characterized by the ability 
to: stay calm and collected 
when confronted with 
adversity, frustration, or 
other difficult situations; and 
avoid defensive reactions or 
Liu + 1 iceimgs as a result ol 
others 5 comments. 




Energy 
50 Minutes 


This competency is 
characterized by a preference 
to stay busy, active, and 
avoid inactive events or 
situations. 


Required 


Business Leadership 


Measures the candidate's 
thinking styles. High scorers 
are likely to have or learn 
good planning and 
organizing skills, be 
innovative, consider issues 
from multiple perspectives, 
and create strategies to build 
their business. 


32 


Leadership 
Motivation 


Measures the candidate's 
desire for achievement, 
drive, initiative, energy 
level, willingness to take 
charge, and persistence. 
High scorers are likely to be 
highly motivated to succeed 
and to set challenging goals 
for themselves and others. 


35 




Self-Leadership 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility for actions, 
and tolerate stress. High 
scorers are also likely to 
have a positive attitude, be 
optimistic about the future, 
and demonstrate high levels 
of profes sionalisrn . 


34 


Selection 35 - 50 Minutes (cont.) 



39 
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Interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 
parsuade and influence 
others, gain commitment, 
and build effective 
interpersonal ralationshipa. 
They also have potential to 
develop .skills in thr. areas of 
employee relations, 
coaching, motivating, and 
leading a team. 


41 


Decision 
Making/Problem 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. This competency 
is characterized by the 
ability to solve complex 
problems, and make 
reasunea occisiuns. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effcfrivfcjy 
use verbal reasoning. This 
competency is characterized 
by the ability to verbally 
explain complex information 
to others. 


10 



40 
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Tabic Nine - Managerial Solutions 


| Solution Component 1 Definition I Items 


Sereeninu 1 0 - 20 Minutes 


Required 


Management 
Potential 


Measures potential for 
managerial success across 
industry type and functional 
area. Scores on 
Management Potential are 
derived from candidates* 
responses to questions 
regarding academic and 
social b^Vground, and 
aspirations concerning work. 


10 


Judgment 


Measures potential for 
making good judgments 
about how to effectively 
respond to work situations. 
Scores on Judgment are 
derived from candidates' 
responses to questions 
regarding situations one 
would likely encounter a* a 
manager/supervisor. 


10 


Optional 


Self-Confidence 


This component references: 
belief in one's own abilities 
and skills and a tendency to 
feci competent in several 
areas. 


10 


Decision Making 


Measures potential for 
success as A manager, This 
is done by having applicants' 
make judgments about the 
most effective decisions in 
specific work situations. 
Their potential is determined 
by comparing their response 
profiles to the profiles of 
success ftil managers. 



41 
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Selection 20 - 3 5 - 50 Mins 


Required 


Business Leadership 


Measures the candid fite's 
thinking styles. High scorers 
arc likely to have or lcam 
good planning and 
organizing skills, be 
innovative, consider issues 
from multiple perspectives, 

their business. 


32 


Leadership 
Motivation 


Measures the candidate's 
desire for achievement, 
drive, initiative, energy 
level, willingness to toke 
charge, and persistence. 
High scorers arc likely to be 
highly motivated to succeed 
and to set challenging goals 

lOl UlCnUjBIVeH uIlU OUlvTtP' 


3^ 




Self-Lfcadership 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility for actions, 
and tolerate stress. High 
scorers are also likely ta 
have a positive attitude, bfc 
optimistic about the future, 
and demonstrate high levels 
of professionalism. 


34 

i — . — 


SAlAfttinn 20 - 35 - 50 Mins (cent) 




Interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 
persuade and influence 
others, gain commitment, 
and build effective • 
interpersonal relationships. 
They also have potential to 
develop skills in the areas of 
employee relations, 
coaching, motivating, and 
leading a team. 


41 
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Optional 


Decision 
Making/Problem 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. This competency 
is characterized by the 
ability to solve complex 
problems, and make 
reasoned decisions. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning. This 
competency i,s characterized 
by the ability to verbally 
explain complex information 
to others. 


10 


[00154] 


Tabic Ten - Technical-Professional Solutions 


I Solution Component | Definition I items 


Sanwinirtg R Minutes 


Required 


Dependability 


This competency is 
characterized by: a 
willingness to behave in 
expected and agree upon 
ways; following through on 
assignments and 
commitments; keeping 
promises; and accepting the 
consequences of one's own 

actions. 


40 


Interpersonal Skills 


This competency is indnced 
by a tendency to be plea.iant 
cooperative, and helpful 
when working with others, 
as well as flexible in conflict 
resolution situations. 



43 
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Self-Control 


This competency is 
characterized by the ability 
to: stay calm and collected 
when confronted with 
adversity, frustration, or 
other difficult situations; and 
avoid defensive reactions or 
hurt feelings as a result of 
others* comments. 




Energy 


This competency is 
characterized by a 
preference to stay busy, 
active, and avoid inactive 
events or situations. 


Selection 35-50 Minutes 


Required 


Business Leadership 


Measures the candidate's 
thinking styles. High scorers 
are likely to have or learn 
good planning and 
organizing skills, be 
innovative, consider issues 
from multiple perspectives, 
and create strategies to build 
theii business. 


32 


Leadership 
Motivation 


Measures the candidate's 
desire for achievewnrrt, 
rWw, initiative, energy 
level willingness to take 
charge, and persistence. 
High scorers are likely to be 
highly motivated to succeed 
and to set challenging goals 
for themeelves and others. 


35 




Self-Leadership 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility for actions, 
and tolerate stress. High 

have a positive attitude, he 
optimistic about the future, 
and demonstrate high levels 
of professionalism. 


34 


Selection 35 — 50 Minutes (cont) 
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Interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 
persuade and influence 
others, gain commitment, 
and build effective 
interpersonal relationships. 
They also have potential to 
develop skills in the HTr%%% nf 
employrn relations, 
coaching, mntivnting, and 
leading a team. 


41 


Decision 
Making/^Pmhlcm 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. Thia competency 
is characterized by the 
ability to solve complex 
problems, and make 
reasoned decisions. 


10 


Optional 


Communicatioa 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning and 
effectively use verbal 
reasoning. This competency 
is characterized by the 
ability to verbally explain 
complex information to 
others. 


10 
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[00155] 



Table Eleven - Executive Solutions 


Solution Component | Definition | Items 


Screening 20 Minutes 


Required 


Executive Potential 


Measures the potential for 
success in high-level 
organizational positions 
across industry type and 
functional area. Scores on 
Executive Potential are 
derived from candidates 5 
response? to questions 
regarding work background, 
accomplishments, and career 
aspirations. 


53 


Selection 35 - 50 Minutes 


Required 


Business Leadership 


Measures the candidate's 
thinking styles. High scorers 
are likely to have or learn 
good planning and 
organizing skills, he 
innovative, consider issues 
from imdtiplr perspectives, 
and create strategics to build 
their business 


32 


Leadership 
Motivation 


Measures the candidate's 
desire for achievement, 
drive, initiatives energy 
level willingness to take 
charge, and persistence. 
High scorers are likely to be 
highly motivated to succeed 
and to set challenging goals 
for themselves and others. 


35 




Self-Leadership 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility for actions, 
and tolerate stress. High 
scorers are also likely to 
have a positive attitude, be 
optimistic about the future, 
and demonstrate high levels 
of professionalism. 


34 
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Interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 

others, gain commitment, 
and build effective 
interpersonal relationships. 
They also have potential to 
develop skills in the areas of 
employee relations, 
coaching, motivating, and 

lEd.UllJj£ a. 1 Colli. 


41 




Decision 
Making/Problem 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
rrflsrvniTTg. Th\s rrrnipetfrnny 
is characterized by the 
ability to solve complex 
problems, and make 
reasoned decisions. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning. This 
competency is characterized 
by the ability to verbally 
explain complex information 
to others. 


10 


[00156] 


Table Twelve -Campus Recruiting Solutions 




Solution Component | Definition 


| Items 


Screening 12 Minutes 


Required 


Supervisory 
Potential 


Measures potential for 
supervisory success across 
industry type and functional, 
area. Scotcs on Supervisory 
Potential are derived from 
candidates 1 responses to 
questions regarding 
academic and social 
background, and aspirations 
concerning work. 


76 
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Judgment 


Measures potential for 
m?ifrm£ good judgments 
ahouthowtn effectively 
respond to work situations. 
Scores on Judgment are 
derived from candidates' 
responses to questions 
regarding situations one 
would likely encounter as a 
manager/supervisor. 




Management 
Potential 


Measures potential for 
managerial success across 
industry type and fiinctjoml 
area. Score* on 
Management Potential tit 
derived from candidates' 
responses to questions 
regarding academic and 
BOoiaJ background, and 
aspirations concerning work. 


Selection 20 - 35 - 50 Mins 


Required 


Business Leadership 


Measures the candidate's 
thinking styles. High scorers 
are likely to have or learn 
good planning and 
organizing skills, br 
innovative, consider issues 
from multiple perspectives, 
and create strategies to build 
their business. 


32 


Leadership 
Motivation 


Measures the candidate's 
desire for achievement, 
drive, initiative, energy 
level, willingness to take 
charge, and persistence. 
High scorers are likely to be 
highly motivated to succeed 
and to set challenging goals 
for themselves and others. 


35 
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Sdf-Leaderrfihip 


Measures the candidate's 
ability to control emotions, 
act with integrity, take 
responsibility foi actions, 
and tolerate stress. High 
scorers are also likely to 
have a positive attitude, be 
optimistic about the future 3 
and demonstrate high levels 
of professional ism. 


34 


Selection 20 - 35 - 50 Mins (corn.) 




interpersonal 
Leadership 


Measures the candidate's 
interpersonal characteristics. 
High scorers are likely to 
prxsuttta and influence 
others, gain crrmmitmcnt, 
and build effective 
interpersonal rdationshipa. 
They also have potential to 
develop skills in the area of 
employee relations, 
coaching, motivating, and 
leading a team. 




Optional 


Decision 
Making/Problem 
Solving 


Measures the tendency to 
efficiently and effectively 
use numerical and analytical 
reasoning. This eanrpetanry 
is c.lmrTirtcriTttri by the 
ability to solve complex 
problems, and make 
reasoned decisions. 


10 


Optional 


Communication 


Measures the tendency to 
efficiently and effectively 
use verbal reasoning. This 
competency is characterized 
by the ability to verbally 
explain complex information 
to others. 


10 
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[00157] 





Tabic Thirteen - 


- Communication Solution 






Solution Component 


Definition 1 


Items 


Selection 37 Minute* 


Required 


Listening 
Orientation 


Measure of the tendency to 
listen to and understand 
others 7 perspectives, to care 
for otherB, to accept and 
respect the individual 
differences of people, and to 
be open both to multiple 
ideas and to using alternative 
mode of thinking. 


73 




English Language 
Skills 


Measures usage of verb 
tense and sentence 
rrmstruction. Scores on 
English Language Skills are 
derived irom candidates 
responses to grammar 
questions. 






Verbal 
Masoning/Critical 
Thinking 


Measures verbal reasoning 
skills and critical thinking/ 
reasoning skills. Scores on 
Verbal Reasoning Ability 
are derived from candidates' 
responses to analogies and 
questions about inf nrmati on 
provirlni in "brief reading 
passages. 





so 
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[00158] 



Tabic Fourteen- Series Bix/Seven Succe&s Solution 






Solution Component 


Definition 




Items 




Screening 


36 Minutes 










Required 




Problem Solving 


Measures the ability to 
analyze and evaluate 
information. Scores on 
Problem Solving are derived 
from candidates' responses 
to mathematical and 
analytical reasoning items, 
requiring candidates to 
rr.sprmsc to facts and figures 
presented in various formats. 




20 








Verbal 
Reasoning/ Critical 
Thinking 


Measures verbal reasoning 
skills and critical 
thinking/reasoning skills. 
Scores on Verbal Reasoning 
Ability are derived from 
candidates' responses to 
analogies and involves 
making inferences from 
information provided in fhft 
form nf brief passages 








[00159] 


Table Fifteen - Information Technology Aptitude Solution 






Solution Component 1 Definition 


1 


Items 






18 Minutes 










Required 


Critical Thinking 


Measure reasoning and 
critical thinking skills. 
Scores on Critical Thinking 
are derived fmm candidates' 
responses to information 
provided in the form of brief 
passages. 


58 
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Problem Solving 



Communication 



Spatial Ability 



Measure the ability to 
analyze and evaluate 
information. Scores On 
Problem Solving are derived 
from candidates' responses 
to mathematical and 
analytical reasoning items, 
requiring candidates to 
respond to facts and figures 
presented in various 
scenarios, 



Measures the ability to 
efficiently use verbal 
information. Scores on 
Ccrrnmunicatinnarc derived 
from candidates' ability to 
identify synonyms, 



Measure the ability to 
visually manipulate otgecta. 
Scores on Spatial Ability are 
derived from candidates 7 
ability to oorrectly identify 
the number of blocks in 
progressively difficult 



[00160] Although the disclosure has focused on recruiting applications, the generated data 
may be used in other human capital applications. FIG. 1 3 illustrates a human capital management, 
life-cycle- Measurement and data 1301 is initially used in the context of recruiting 1302. For 
recruiting 1302, screening, selecting, And interview solutions measure applicants 1 competencies 
and predict on-the-job performance and thus contribution to business outcomes. 
[00161] For compensation 1303, data about potential can be -weighed againirt performance 
data to ensure that high potential employees who arc on difficult assignments where they are 
structurally constrained from succeeding axe not underpaid by pure focus on performance. For 
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example, structural constraints may include business environments, poor staff, unreliable 
equipment, etc. 

[00162] For retention 1304, businesses with jobs that high turnover use the system to 
ensure that applicants have, qualities that contribute to longer tenure in roles. 
(00163] For performance role 1305, the sy$tftm van. be used to enhance the validity of 
employee performance evaluation. 

[00164] For training and development 1306, * rnmpany may test current employees in 
order to design executive training programs addressing each individual's strengths and 
weaknesses. Or, for employees that took a test and were hired despite weaknesses, me data can 
bp iwH to structure appropriate training. 

[00165] Fnr succession 1307, data on employees may be collected in the process of 
organisation mergers to assist planning for retrenchment or change. Also, by measuring 
compentencies and mapping them between roles, it is possible to assess the potential that an 
individual may have for a role other than the job they are cwrcntly holding, such as for a 
promotion or a Transfer to another area. 

[00166] The foregoing description is to be considered as illustrative only. The skilled 
artisan will recognize many variations and peimutfitjnns within the spirit of the disclosure. 
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Abstract 

[00167] A system and method for testing and/or evaluating employees or potential 
employees is disclosed. A computer arranges a plurality of applicants in a stack ranked table 
The table may rank or re-rank applicants against each other, from beat to worst, after successive 
screening, selecting, and/or interviewing stages for a particular job. Performance evaluations of 
hired workers may be fed back to the computer for adjusting the system and method. 
Competencies shown to be predictive of successful performance of a given type of job are tested 
for at various stages in an online testing system. 
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